基于深度学习的视频跟踪研究进展综述

近年来深度学习迅猛发展，颠覆了语音识别、图像分类、文本理解等领域的算法设计思路。深度学习因其具备强大的特征提取能力，在图像识别领域的成绩尤为突出。然而深度学习与视频监控领域的结合并不多，由于深度模型具有多层网络结构，算法复杂度大，训练和更新模型时比较耗时，很难满足实时性要求。回顾了深度学习的发展史，介绍了最近10年来国内外深度学习主要模型，论述了基于深度学习的目标跟踪算法，指出了各算法的优缺点，最后对当前该领域存在的问题和发展前景进行了总结和展望。     

基于深度卷积长短时神经网络的视频帧预测

针对视频帧预测中难以准确预测空间结构信息细节的问题，通过对卷积长短时记忆（LSTM）神经网络的改进，提出了一种深度卷积长短时神经网络的方法。首先，将输入序列图像输入到两个不同通道的深度卷积LSTM网络组成的编码网络中，由编码网络学习输入序列图像的位置信息变化特征和空间结构信息变化特征；然后，将学习到的变化特征输入到与编码网络通道数对应的解码网络中，由解码网络输出预测的下一张图；最后，将这张图输入回解码网络中，预测接下来的一张图，循环预先设定的次后输出全部的预测图。与卷积LSTM神经网络相比，在Moving-MNIST数据集上的实验中，相同训练步数下所提方法不仅保留了位置信息预测准确的特点，而且空间结构信息细节表征能力更强。同时，将卷积门控循环单元（GRU）神经网络的卷积层加深后，该方法在空间结构信息细节表征上也取得了提升，检验了该方法思想的通用性。     

A signal adaptive diffusion filter for video coding: Mathematical framework and complexity reductions

In this paper we combine video compression and modern image processing methods. We construct novel iterative filter methods for prediction signals based on Partial Differential Equation (PDE) based methods. The mathematical framework of the employed diffusion filter class is given and some desirable properties are stated. In particular, two types of diffusion filters are constructed: a uniform diffusion filter using a fixed filter mask and a signal adaptive diffusion filter that incorporates the structures of the underlying prediction signal. The latter has the advantage of not attenuating existing edges while the uniform filter is less complex. The filters are embedded into a software based on HEVC with additional QTBT (Quadtree plus Binary Tree) and MTT (Multi-Type-Tree) block structure. In this setting, several measures to reduce the coding complexity of the tool are introduced, discussed and tested thoroughly. The coding complexity is reduced by up to 70% while maintaining over 80% of the gain. Overall, the diffusion filter method achieves average bitrate savings of 2.27% for Random Access having an average encoder runtime complexity of 119% and 117% decoder runtime complexity. For individual test sequences, results of 7.36% for Random Access are accomplished.     

Self-learning residual model for fast intra CU size decision in 3D-HEVC

As an extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC requires to encode multiple texture views and depth maps, which inherits the same quad-tree coding structure as HEVC. Due to the distinct properties of texture views and depth maps, existing fast intra prediction approaches were presented for the coding of texture views and depth maps, respectively. To further reduce the coding complexity of 3D-HEVC, a self-learning residual model-based fast coding unit (CU) size decision approach is proposed for the intra coding of both texture views and depth maps. Residual signal, which is defined as the difference between the original luminance pixel and the optimal prediction luminance pixel, is firstly extracted from each CU. Since residue signal is strongly correlated with the optimal CU partition, it is used as the feature of each CU. Then, a self-learning residual model is established by intra feature learning, which iteratively learns the features of the previously encoded coding tree unit (CTU) generated by itself. Finally, a binary classifier is developed with the self-learning residual model to early terminate CU size decision of both texture views and depth maps. Experimental results show the proposed fast intra CU size decision approach achieves 33.3% and 49.3% encoding time reduction on average for texture views and depth maps with negligible loss of overall video quality, respectively.     

Content adaptive video denoising based on human visual perception

In this paper, we propose content adaptive denoising in highly corrupted videos based on human visual perception. We introduce the human visual perception in video denoising to achieve good performance. In general, smooth regions corrupted by noise are much more annoying to human observers than complex regions. Moreover, human eyes are more interested in complex regions with image details and more sensitive to luminance than chrominance. Based on the human visual perception, we perform perceptual video denoising to effectively preserve image details and remove annoying noise. To successfully remove noise and recover the image details, we extend nonlocal mean filtering to the spatiotemporal domain. With the guidance of content adaptive segmentation and motion detection, we conduct content adaptive filtering in the YUV color space to consider context in images and obtain perceptually pleasant results. Extensive experiments on various video sequences demonstrate that the proposed method reconstructs natural-looking results even in highly corrupted images and achieves good performance in terms of both visual quality and quantitative measures.     

E?cient Video Cutout by Paint Selection

Video cutout refers to extracting moving objects from videos, which is an important step in many video editing tasks. Recent algorithms have limitations in terms of e?ciency, interaction style, and rob...     

Optimal-correlation-based reconstruction for distributed compressed video sensing

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient joint reconstruction by leveraging more realistic signal models is still an open challenge. In this paper, we present a novel optimal-correlation-based reconstruction method for compressively sampled videos from multiple measurement vectors. In our method, the sparsity is mainly exploited through inter-signal correlations rather than the traditional frequency transform, wherein the optimization is not only over the signal space to satisfy data consistency but also over all possible linear correlation models to achieve minimum-l₁-norm correlation noise. Additionally, a two-phase Bregman iterative based algorithm is outlined for solving the optimization problem. Simulation results show that our proposal can achieve an improved reconstruction performance in comparison to the conventional approaches, and especially, offer a 0.7–9.9 dB gain in the average PSNR for DCVS.     

A self-adaptive weighted affinity propagation clustering for key frames extraction on human action recognition

In this paper, we propose a novel approach for key frames extraction on human action recognition from 3D video sequences. To represent human actions, an Energy Feature (EF), combining kinetic energy and potential energy, is extracted from 3D video sequences. A Self-adaptive Weighted Affinity Propagation (SWAP) algorithm is then proposed to extract the key frames. Finally, we employ SVM to recognize human actions on the EFs of selected key frames. The experiments show the information including whole action course can be effectively extracted by our method, and we obtain good recognition performance without losing classification accuracy. Moreover, the recognition speed is greatly improved.     

Rare Titin (TTN) Variants in Diseases Associated with Sudden Cardiac Death

A leading cause of death in western countries is sudden cardiac death, and can be associated with genetic disease. Next-generation sequencing has allowed thorough analysis of genes associated with this entity, including, most recently, titin. We aimed to identify potentially pathogenic genetic variants in titin. A total of 1126 samples were analyzed using a custom sequencing panel including major genes related to sudden cardiac death. Our cohort was divided into three groups: 432 cases from patients with cardiomyopathies, 130 cases from patients with channelopathies, and 564 post-mortem samples from individuals showing anatomical healthy hearts and non-conclusive causes of death after comprehensive autopsy. None of the patients included had definite pathogenic variants in the genes analyzed by our custom cardio-panel. Retrospective analysis comparing the in-house database and available public databases also was performed. We identified 554 rare variants in titin, 282 of which were novel. Seven were previously reported as pathogenic. Of these 554 variants, 493 were missense variants, 233 of which were novel. Of all variants identified, 399 were unique and 155 were identified at least twice. No definite pathogenic variants were identified in any of genes analyzed. We identified rare, mostly novel, titin variants that seem to play a potentially pathogenic role in sudden cardiac death. Additional studies should be performed to clarify the role of these variants in sudden cardiac death.     

Capturing Preservice Chemistry Teachers’ Visual Representations of Redox Reactions through Storyboards

This study aimed to identify preservice chemistry teachers’ understanding of redox reactions at the submicroscopic level and how they make connections to macroscopic level. Twenty‐five preservice chemistry teachers first viewed a video of a redox reaction happening between a zinc wire and a copper(II) sulfate solution, then they reflected on their macroscopic observations, and generated storyboards (a sequence of drawings) representing this reaction at the submicroscopic level. Both the reflections and the storyboards were coded, and the categories reflecting the participants’ levels of understanding were determined. Results showed that the participants usually identified the macroscopic evidences, but they had conceptual difficulties in interpreting and representing evidences at the submicroscopic level. Few participants indicated a scientific or a moderate understanding of redox reactions, whereas the majority exhibited either weak or alternative understanding of redox reactions.